This application claims the benefit of Application No. 09-098236, filed in Japan on Mar. 31, 1997, which is hereby incorporated by reference.
1. Field of the Invention
The present invention relates to a solid-state imaging device and, in particular, to an apparatus that can be used for industrial robots or a variety of inspection devices. By employing functions that conduct the operation of summing the product of image signals from the imaging unit of a solid-state imaging device, it is possible to capture an image from the imaged data of a subject as well as output the image data after processing, without using a high-performance processor and without imposing an excessive burden on secondary processors due to the image processing.
2. Discussion of the Related Art
Currently, image processing devices used for industrial robots and various inspection devices have been designed such that image capturing of the object to be measured is implemented using a solid-state imaging device, such as a CCD. The captured image signal or data is sent to a high-performance processor that sequentially conducts various operations on the imaged data. The solid-state imaging device is generally only for capturing the image, and normally the captured data from the solid-state imaging device is processed by secondary processing devices.
Unfortunately, the number of picture elements of a solid-state imaging device ranges from the tens of thousands to several hundred thousand, and may even reach several million picture elements/frame. Therefore, the amount of image data obtained from the picture elements is enormous. Thus, in order to conduct processing for this enormous amount of image data in real time, it is necessary to have extremely high capacity processing. Also, it is necessary to substantially increase the transmission speed of the data between the CCD, memory, and the processor as well as increasing the processing speed of the processor. A drawback has been of overly burdening of the systems. Most notably the processor was over-burdened. As a result of trying to address these drawbacks, the size of the apparatus was increased, which also increased the costs to produce it.
One method employed to solve the attendant drawbacks is to use two processors, rather than one, to conduct the processing of the image and other processing. One processor may be dedicated to image processing only. This is an effective method when the target object of the image processing is limited to the picture element under consideration and its periphery, such as for differential processing. However, when the entire image area is the target object, such as for a fast Fourier transformation (FFT), even designating a processor for image processing only results in inadequate performance. Additionally, there are many applications wherein it is difficult to avoid the imaging apparatus from becoming excessively large and expensive.
In recent years there have been attempts to integrate an arithmetic circuit for image processing into the solid-state imaging device. However, because of the multitude of devices being employed for operations in the picture element unit, as the size of the chip of the solid-state imaging device increases the aperture number decreases, resulting in deterioration in the image sensing performance.
The object of the present invention is to obviate one or more of the above problems of the solid-state imaging devices of the prior art and to implement product summing of the picture element unit of the solid-state imaging device with a simple circuit structure, and to conduct high-speed image capturing of the subject as well as processing without placing an excessive burden on the processors due to picture processing.
Accordingly, the present invention is directed to a solid-state imaging device that substantially obviates one or more of the problems due to limitations and disadvantages of the related art.
An object of the present invention is to provide a solid-state imaging device with a simple apparatus structure that is capable of conducting all operations from image capturing to image processing.
Another object of the present invention is to provide a solid-state switching device capable of turning on only one of a plurality of readout lines simultaneously.
A further object of the present invention is to provide a solid-state imaging device that can conduct high-speed arithmetic processing.
A still further object of the present invention is to provide a solid-state imaging device that can conduct high-speed arithmetic processing without an external processor.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
To achieve these and other advantages and in accordance with the purpose of the present invention, as embodied and broadly described, the solid-state imaging device includes a plurality of amplifying picture elements for converting and amplifying optical signals and outputting the optical signals, at least one readout line for reading out signals from a predetermined number of picture elements among the plurality of amplifying picture elements, and a plurality of switching devices connected between each output of the plurality of amplifying picture elements and the corresponding readout lines.
In another aspect, the solid-state imaging device includes solid-state imaging device includes multiple picture elements wherein each of the picture elements includes a photodetector that converts optical signals into an electrical charge and stores it, an amplifying device that amplifies the electrical charge signal that is stored by the photodetector, a transfer device that transfers the electrical charge signal that is stored by the photodetector to a control electrode of the amplifying device, and a resetting device that resets the electrical charge of the control electrode of the amplifying device, wherein the multiple picture elements are arranged in an orientation of rows and columns along two dimensions and conduct a signal readout as a current signal, multiple vertical readout lines provided on each of the columns in order to readout signals from the picture elements that are arranged on each corresponding column, multiple switching devices wherein each switching device corresponds to each of the picture elements and is connected between the output of the amplifying device of the picture element and the corresponding vertical readout line, a horizontal readout line that outputs the signal from the multiple vertical readout lines, and horizontal switching devices in correspondence to each of the vertical readout lines and supplies signals from each vertical readout line to the horizontal readout line. In still another aspect, the solid-state imaging device includes an exposure method, wherein the solid-state imaging device exposure method includes the steps of turning off the transfer devices of all of the picture elements during the designated exposure time and transferring the electrical charge of the photodetector devices to the control electrode of the amplifying devices by turning on the transfer devices of all the picture elements again, detecting signals with a designated frequency component by letting n be a positive integer and finding a first sum total by conducting a simultaneous read out of n-rows at a time, skipping every other n-rows, or n-columns at a time skipping every other n-columns, determining the second sum total by conducting a simultaneous read out of n-rows at a time skipping every other n-rows between the n-rows at a time that were read out, or n-columns at a time skipping every other n-columns between the n-columns at a time that were read out, and determining a difference between the first and the second sum totals by using the differential circuit.
In the solid-state imaging device an exposure is conducted by turning off the transfer devices of all the picture elements during the designated exposure time, and transferring the electrical charge of the photodetector devices to the control electrode of the amplifying devices by turning on the transfer devices of all the picture elements again. The signals with the designated frequency component are then detected by letting n be a positive integer and finding the first sum total by conducting a simultaneous read out of n-rows at a time, skipping every other n-rows at a time, or n-columns at a time, skipping every other n-columns at a time. The second sum total is found by performing a simultaneous read out of n-rows at a time, skipping every other n-rows between the n-rows at a time that were read out, or n-columns at a time, skipping every other n-columns at a time, between the n-columns at a time that were read out. The difference is then determined between the first and second sum totals by using the differential circuit.
This configuration makes it possible to select individual amplifying image elements and to output the image elements on a readout line as a current signal using the same method as the solid-state imaging devices disclosed in the prior art. It is possible to conduct various types of image processing inside the solid-state imaging device by compositing the output signals from the amplifying image elements or calculating a sum of the products by controlling the switching device connected between the output of the amplifying image elements and the corresponding readout line. By processing the image data inside the solid-state imaging device, without placing too great of a burden on the processor for image processing, it is possible to achieve a solid-state imaging device with a simple apparatus structure that is capable of conducting all operations from image capturing to image processing.
It is advantageous for the switching devices that are to be connected to the same readout line to be configured so that it is possible to turn on only one or a plurality of readout lines simultaneously. If a plurality of switching devices are turned on, the picture elements are displayed as a composite of the plurality of the readout lines on the readout line.
If only one of the switching devices is turned on at a given time, it is possible to obtain an individual output signal from each picture element. If a plurality of the switching devices are simultaneously turned on, the output of the plurality of the picture elements can be displayed as a composite signal on the readout line, where it is possible to perform summing of the output of a plurality of picture elements. Therefore, without conducting image processing by an external processor, the solid-state imaging device can conduct high-speed arithmetic processing.
The above designs allow the sequentially read out signals from each of the picture elements, via a vertical readout line and a horizontal readout line, as is done with the solid-state imaging device of the prior art. It is possible to output signals from multiple picture elements by converting them into a composite signal, on a vertical readout line or a horizontal readout line, by controlling a switching device that is connected between the output of the amplifying device of the picture elements and their corresponding vertical readout line. Thus, by using the design of the present invention, the operation of summing products can be conducted for the output signal of multiple picture elements inside the solid-state imaging device. Consequently, high-speed picture processing can be implemented with a simple circuit design. Moreover, the processor designated for image processing only is not excessively burdened due to image processing performed in this manner. The circuit configuration is simple because only a switching device is added to the picture element unit of the solid-state imaging device of the prior art and the drawbacks, such as the enlarging of the chip size or the decreasing of the aperture number, is prevented.
By additionally providing a current/voltage conversion circuit, connected to the horizontal readout line, a read out voltage output is obtained via the current/voltage conversion circuit. By connecting the current/voltage conversion circuit to the horizontal readout line, it is possible to accept output signals that are read out from the picture elements as current signals via the vertical readout line and the horizontal switching device and obtain the corresponding voltage output. This configuration makes it possible to obtain a voltage signal as the output signal while taking advantage of the current readout mode.
For the switching devices connected to the same vertical readout line, it is possible to turn on only one or a plurality of readout lines simultaneously, and it is possible to configure the circuit such that if a plurality of switching devices are simultaneously turned on the output of the plurality of picture elements can be converted and displayed as a composite on the vertical readout line.
For the switching devices connected to the same vertical readout line, at a time when only one switching device is turned on, it is possible to output a signal individually from each picture element onto the vertical readout line. Furthermore, when a plurality of the switching devices are turned on simultaneously, the sum of the products operation for the output of a plurality picture elements is conducted by making a composite of the output of a plurality picture elements on the vertical readout line.
Moreover, for the horizontal switching devices it is possible to have a configuration that only one or a plurality of switching devices can be turned on, and if a plurality of switching devices are turned on the outputs of the plurality of switching devices of the picture elements are converted into a composite of the outputs and the composite output is displayed on the horizontal readout line.
If only one horizontal switching device is turned on at a given time, it is possible to output an individual output signal from each picture element from the horizontal readout line. On each vertical readout line, it is possible to output picture elements output from each row sequentially. By simultaneously turning on a plurality of switching devices connected to the vertical readout line, it is possible to output the sum of the products operation for the output of the picture elements form a plurality of rows by simultaneously turning on a plurality of switching devices connected to the vertical readout line. This allows the creation of a composite of signals from more than one picture element in the same row and different columns on the horizontal readout line, or the creation of a composite of signals from a plurality of picture elements in different rows and the same column on the horizontal readout line.
Additionally, it is advantageous to also incorporate a differential circuit into the design that is capable of finding the difference between the signals that are sequentially read out on the horizontal readout line.
By connecting the differential circuit to the horizontal readout line, the difference between the signals that are read out sequentially are determined, as well as the difference between the signals obtained by calculating the sum of the products output for each of a plurality of picture elements.
The present invention also provides for the capability of having at least a primary vertical scanning circuit that supplies a selection control signal for picture elements that are provided on every row and a secondary vertical scanning circuit that supplies signals to control the switching devices.
Due to the provision of providing a primary vertical scanning circuit that supplies a selection control signal for each picture element that is provided on every row, which is similar to the prior art, and a secondary vertical scanning circuit that controls the switching devices, the present invention is capable of performing operations that are similar to those conducted by the solid-state imaging devices of the prior art that were mainly performed by the primary vertical scanning circuit. The secondary scanning circuit operations, such as the summing of products, are performed done by controlling the switching devices. By doing so, the structure of the peripheral circuit of the solid-state imaging device is simplified and control is facilitated.
An exposure can be performed by turning off the transfer devices for all the picture elements during the designated exposure time. The electrical charge of the photodetector device is transferred to the control electrode of the amplifying device by turning on the transfer device of all the picture elements again. A batch exposure operation is then performed with the shutter function by selecting picture elements with the switching device from every row reading out the signals from each column to each vertical readout line. The signals are then sequentially outputted from each vertical readout line to the horizontal readout line, in series, using the horizontal switching devices.
This structure allows for the simultaneous transfer of the electrical charges of all the picture elements from the transfer device to the control electrode of the amplifying device. The switching device is then used to read out the signals of each picture element onto each vertical readout line, as well as to output the signals of each vertical readout line in a time sequence by using the horizontal switching devices. The present invention is capable of transferring the electrical charges of the photodetector to the control electrode in advance of the exposure. The electrical charges can also be outputted to the resetting device by turning on the transfer devices of all the picture elements. This allows for a simultaneous resetting of all the picture elements and a simultaneous transferring of the electrical charges of all the picture elements afterward. Therefore, it is possible to operate the solid-state imaging device as an image sensor for a batch exposure having a shutter function, as in a CCD device.
An exposure is capable of being performed in the present invention by turning off the transfer devices of all the picture elements during the designated exposure time and transferring the electrical charge of the photodetector device to the control electrode of the amplifying device by turning on the transfer device for all the picture elements again. A composite of the electrical charge of the picture elements in each picture element area across multiple rows and multiple columns is then read out sequentially.
The compositing of the electrical charges of the picture elements in each picture element area across multiple rows and multiple columns and being read out sequentially, allows for the removal of noise by averaging the signals of the multiple picture elements. Therefore, although the resolution decreases, the sensitivity increases and an image of a dark subject with reduced noise can be obtained.
The present invention further allows for and contemplates that a portion of each of the picture element areas that are sequentially read out overlap each other. By having a portion of each of the picture element areas overlapping each other, an image can be read out with reduced noise while maintaining the horizontal and/or vertical scanning speed at a designated value.
Additionally, it is possible to have a structure wherein the picture element areas that are sequentially read out are shifted by one picture element. If the picture element areas that are sequentially read out are shifted by one picture element, it is possible to have a horizontal and/or vertical scanning speed that is the same as when a picture element area is read out one picture element at a time. The displaying of the picture or recording can be easily conducted using a typical device.
The picture element areas that are sequentially read out can also be adjacent to each other. By arranging the picture element areas that are sequentially read out so that they are adjacent to each other, improved frame speed can be obtained so that a high speed read out can be performed. Because the outputs of more than one picture element are composited and read out, even when this kind of high speed readout is conducted, the total amount of electrical charge increases and consequently the sensitivity does not decrease. In other words, it is possible to carry out high speed operations as well as maintain a high sensitivity readout.
An exposure can be performed by turning off the transfer devices of all the picture elements during the designated exposure time and transferring the electrical charge of the photodetector devices to the control electrode of the amplifying devices by turning on the transfer devices of all the picture elements again. The signals of the designated frequency component can then be detected by letting n be a positive integer and finding the first sum total by conducting a simultaneous readout of n-rows at a time, or every other n-rows, or n-columns at a time, or every other n-columns. The second sum total can then be found by conducting a simultaneous readout of n-rows at a time skipping every other n-rows between the n-rows at a time that were read out, or n-columns at a time skipping every other n-columns between the n-columns at a time that were read out. The difference between the first and second sum totals is then found using the differential circuit.
The present circuit structure is capable of detecting signals having each frequency component in the vertical or horizontal direction of the image. For example, by letting n=1, the maximum frequency in the vertical or horizontal direction is detected. When n=2, the frequency component at xc2xd of the maximum frequency is detected. When n=3, the frequency component at ⅓ of the maximum frequency is detected. In other words, signals at the desired frequency component for the image can be easily detected without using an additional filter or signal processor. Therefore, it is possible to take a Fourier transformation of the spatial image through the detection of signals with various frequency components obtained by calculating the sum totals of the first and second sum totals and finding the difference between them for various values of n.
By setting n to various values, detecting signals with various frequency components and adding them, a Fourier transformation of the spatial image can easily be performed. The implementation of a Fourier transformation with a simple device circuit structure of the present invention makes it unnecessary to use a signal processor or filter.
The projection of each row can be obtained by summing the total of the picture elements for each row by selecting signals from all the columns using the horizontal switching device as well as by reading out by sequentially selecting each row. The projection of each column similarly can be obtained by adding the signals from the picture elements of all the rows using the switching device and obtaining the sum total of the signals for each column on the vertical readout line, and then sequentially reading out the signals for each column using the horizontal switching device.
This configuration is capable of obtaining a projection that is the addition of the components of each row or each column of the image, representing a type of characteristic value of the image. Therefore, a special image processing device is not necessary for the solid-state imaging device of the present invention to easily obtain the projection of each row or each column.
The solid-state imaging device of the present invention is also capable of conducting a readout and signal processing for only the picture elements on a portion of the screen using the switching device and the horizontal switching devices.
The solid-state imaging device is capable of conducting a resetting of the picture elements and a transfer of the signals from all the picture elements and is further capable of conducting a readout for only a portion of the picture elements by using the switching device and the horizontal switching devices. Thus, operations and processing for only a portion of the picture elements can be performed. Therefore, it is possible to effectively obtain only the necessary image information. Additionally, the resetting of the picture element portion can be conducted individually and separately from the readout by the switching devices. Therefore, with partial scanning, by conducting a resetting of unnecessary picture elements during the non-readout time, the solid-state imaging device is able to prevent the so-called blooming and smearing effects, which are due to the saturation of non-used picture elements.
The solid-state imaging device of the present invention is capable of performing the operation of summing the product of the signals from the picture elements by using the switching device as a variable resistance device.
The switching device can be a MOS transistor or similar device, wherein the voltage of the control signal applied to the gate is not a digital signal, which has only two values, but rather is an analog signal. By utilizing the analog signal the switching device can be used as a variable resistance device. Therefore, by outputting the signal from each picture element with a change in the gain a weighted summing of products can easily be performed.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.